Add fused recurrent_kda for decode and inference prefill - #317

## Human Note

## Agent note

This is the missing public entrypoint that torchtitan's KDA layer (pytorch/torchtitan#4164) expects
for its "recurrent" backend. `recurrent_kda(q, k, v, gate, beta, initial_state=None, *,
cu_seqlens=None, output_final_state=False)` mirrors `chunk_kda`'s signature and packed contract:
device-resident int32 offsets, repeated offsets as empty padding slots whose state passes through
bitwise, a terminal offset below physical capacity leaving the tail outside the contract, and FP32
recurrent states with one leading entry per logical sequence. The distinction from the chunked core
is encoded in the argument name: `gate` is the per-token log2 decay (`bounded_gate_cumsum` with
`chunk_size=1`), not the chunk-local cumulative gate. Queries scale by 1/sqrt(K) internally,
matching `chunk_kda` and the `naive_recurrent_kda` default.

The Triton kernel scans tokens sequentially per (sequence, head, value-block) program with the
[K, BV] FP32 state held in registers, computing in FP32 regardless of input dtype. It is
inference-only: gradient-requiring inputs are rejected with a pointer at `chunk_kda` rather than
silently detaching. The op uses the repo's define/impl + fake registration with a fixed-arity
(output, state) schema, and joins the lazy `KDA_OPS` surface in `attn_gym.linear`.

## Test Plan

```bash
gpu-run auto -- ~/.venvs/ag-linear/bin/python -m pytest test/test_kda_recurrent.py -q
# 14 passed on B200: dense/packed vs naive_recurrent_kda (fp32 exact-tolerance, bf16),
# non-power-of-two K/V masking, empty-slot state passthrough, capacity tails,
# cross-check vs chunk_kda via bounded_gate_cumsum(chunk_size=1 vs 64), gradient
# rejection, opcheck, fullgraph compile, and CUDA-graph replay with mutated
# boundaries and history.
```

stack-info: PR: #317, branch: drisspg/stack/79

Inference callers currently normalize q with a standalone l2norm pass before chunk_kda, which reads
and writes the full [T, H, 128] tensor just to apply a per-row scalar. Every q-dependent term in the
forward is linear in q's row and lands in the same output row, so the norm can be deferred: the
grams stay in the raw-q basis (intra and K3b unchanged), the output kernel accumulates each row's
sum of squares inside its existing k-tile loop, and one 1/||q|| row scale on the accumulated output
completes the normalization. An earlier variant that emitted an rstd tensor from the intra kernel
cost +154us from register pressure in that 1-warp kernel and was rejected. The fused ragged output
launch needs a slightly larger register cap (152 vs 136) for the extra accumulator.

The flag is opt-in and forward-only: it raises when any input requires grad (the saved Aqk stays in
the raw-q basis, so the existing backward would be wrong) and when compiling (the fused route
bypasses the compiler-opaque autograd op). This mirrors the precedent elsewhere: FLA's
use_qk_l2norm_in_kernel supports backward only because it launches the standalone l2norm kernels
inside the op rather than fusing them, while every true fusion (Helion varlen, FlashInfer decode)
is forward-only. k keeps its explicit l2norm; this complements the decode-side preprocessing fusion
in the recurrent path (#317, #334) on the chunked prefill side. Raw-q grams run about sqrt(K) hotter,
so the flag assumes the bounded-gate contract; the eps floor matches l2norm's default and the tests
cover zero rows and 1e-3..1e3 row scales against the explicit path.

GB300, six frozen contract shapes, output-only forward, warm CUDA events: removes the 38-60us
standalone q pass per call (l2norm kernel eliminated; +5us sumsq cost inside the o kernel). This
flipped the h96/h64 mixed-shape comparisons against Helion's pretuned linear-attention kernels;
final six-shape geomean time ratio ours/helion = 0.952.

```bash
pytest test/test_kda_fuse_q_l2norm.py test/test_kda*.py
python agent_space/bench_three_way.py
```

stack-info: PR: #345, branch: drisspg/stack/84

Inference callers currently normalize q with a standalone l2norm pass before chunk_kda, which reads
and writes the full [T, H, 128] tensor just to apply a per-row scalar. Every q-dependent term in the
forward is linear in q's row and lands in the same output row, so the norm can be deferred: the
grams stay in the raw-q basis (intra and K3b unchanged), the output kernel accumulates each row's
sum of squares inside its existing k-tile loop, and one 1/||q|| row scale on the accumulated output
completes the normalization. An earlier variant that emitted an rstd tensor from the intra kernel
cost +154us from register pressure in that 1-warp kernel and was rejected. The fused ragged output
launch needs a slightly larger register cap (152 vs 136) for the extra accumulator.

The flag is opt-in and forward-only: it raises when any input requires grad (the saved Aqk stays in
the raw-q basis, so the existing backward would be wrong) and when compiling (the fused route
bypasses the compiler-opaque autograd op). This mirrors the precedent elsewhere: FLA's
use_qk_l2norm_in_kernel supports backward only because it launches the standalone l2norm kernels
inside the op rather than fusing them, while every true fusion (Helion varlen, FlashInfer decode)
is forward-only. k keeps its explicit l2norm; this complements the decode-side preprocessing fusion
in the recurrent path (#317, #334) on the chunked prefill side. Raw-q grams run about sqrt(K) hotter,
so the flag assumes the bounded-gate contract; the eps floor matches l2norm's default and the tests
cover zero rows and 1e-3..1e3 row scales against the explicit path.

GB300, six frozen contract shapes, output-only forward, warm CUDA events: removes the 38-60us
standalone q pass per call (l2norm kernel eliminated; +5us sumsq cost inside the o kernel). This
flipped the h96/h64 mixed-shape comparisons against Helion's pretuned linear-attention kernels;
final six-shape geomean time ratio ours/helion = 0.952.

```bash
pytest test/test_kda_fuse_q_l2norm.py test/test_kda*.py
python agent_space/bench_three_way.py
```

stack-info: PR: #345, branch: drisspg/stack/84

Inference callers currently normalize q with a standalone l2norm pass before chunk_kda, which reads
and writes the full [T, H, 128] tensor just to apply a per-row scalar. Every q-dependent term in the
forward is linear in q's row and lands in the same output row, so the norm can be deferred: the
grams stay in the raw-q basis (intra and K3b unchanged), the output kernel accumulates each row's
sum of squares inside its existing k-tile loop, and one 1/||q|| row scale on the accumulated output
completes the normalization. An earlier variant that emitted an rstd tensor from the intra kernel
cost +154us from register pressure in that 1-warp kernel and was rejected. The fused ragged output
launch needs a slightly larger register cap (152 vs 136) for the extra accumulator.

The flag is opt-in and forward-only: it raises when any input requires grad (the saved Aqk stays in
the raw-q basis, so the existing backward would be wrong) and when compiling (the fused route
bypasses the compiler-opaque autograd op). This mirrors the precedent elsewhere: FLA's
use_qk_l2norm_in_kernel supports backward only because it launches the standalone l2norm kernels
inside the op rather than fusing them, while every true fusion (Helion varlen, FlashInfer decode)
is forward-only. k keeps its explicit l2norm; this complements the decode-side preprocessing fusion
in the recurrent path (#317, #334) on the chunked prefill side. Raw-q grams run about sqrt(K) hotter,
so the flag assumes the bounded-gate contract; the eps floor matches l2norm's default and the tests
cover zero rows and 1e-3..1e3 row scales against the explicit path.

GB300, six frozen contract shapes, output-only forward, warm CUDA events: removes the 38-60us
standalone q pass per call (l2norm kernel eliminated; +5us sumsq cost inside the o kernel). This
flipped the h96/h64 mixed-shape comparisons against Helion's pretuned linear-attention kernels;
final six-shape geomean time ratio ours/helion = 0.952.

```bash
pytest test/test_kda_fuse_q_l2norm.py test/test_kda*.py
python agent_space/bench_three_way.py
```

stack-info: PR: #345, branch: drisspg/stack/84

Inference callers currently normalize q with a standalone l2norm pass before chunk_kda, which reads
and writes the full [T, H, 128] tensor just to apply a per-row scalar. Every q-dependent term in the
forward is linear in q's row and lands in the same output row, so the norm can be deferred: the
grams stay in the raw-q basis (intra and K3b unchanged), the output kernel accumulates each row's
sum of squares inside its existing k-tile loop, and one 1/||q|| row scale on the accumulated output
completes the normalization. An earlier variant that emitted an rstd tensor from the intra kernel
cost +154us from register pressure in that 1-warp kernel and was rejected. The fused ragged output
launch needs a slightly larger register cap (152 vs 136) for the extra accumulator.

The flag is opt-in and forward-only: it raises when any input requires grad (the saved Aqk stays in
the raw-q basis, so the existing backward would be wrong) and when compiling (the fused route
bypasses the compiler-opaque autograd op). This mirrors the precedent elsewhere: FLA's
use_qk_l2norm_in_kernel supports backward only because it launches the standalone l2norm kernels
inside the op rather than fusing them, while every true fusion (Helion varlen, FlashInfer decode)
is forward-only. k keeps its explicit l2norm; this complements the decode-side preprocessing fusion
in the recurrent path (#317, #334) on the chunked prefill side. Raw-q grams run about sqrt(K) hotter,
so the flag assumes the bounded-gate contract; the eps floor matches l2norm's default and the tests
cover zero rows and 1e-3..1e3 row scales against the explicit path.

GB300, six frozen contract shapes, output-only forward, warm CUDA events: removes the 38-60us
standalone q pass per call (l2norm kernel eliminated; +5us sumsq cost inside the o kernel). This
flipped the h96/h64 mixed-shape comparisons against Helion's pretuned linear-attention kernels;
final six-shape geomean time ratio ours/helion = 0.952.

```bash
pytest test/test_kda_fuse_q_l2norm.py test/test_kda*.py
python agent_space/bench_three_way.py
```

stack-info: PR: #345, branch: drisspg/stack/84

Inference callers currently normalize q with a standalone l2norm pass before chunk_kda, which reads
and writes the full [T, H, 128] tensor just to apply a per-row scalar. Every q-dependent term in the
forward is linear in q's row and lands in the same output row, so the norm can be deferred: the
grams stay in the raw-q basis (intra and K3b unchanged), the output kernel accumulates each row's
sum of squares inside its existing k-tile loop, and one 1/||q|| row scale on the accumulated output
completes the normalization. An earlier variant that emitted an rstd tensor from the intra kernel
cost +154us from register pressure in that 1-warp kernel and was rejected. The fused ragged output
launch needs a slightly larger register cap (152 vs 136) for the extra accumulator.

The flag is opt-in and forward-only: it raises when any input requires grad (the saved Aqk stays in
the raw-q basis, so the existing backward would be wrong) and when compiling (the fused route
bypasses the compiler-opaque autograd op). This mirrors the precedent elsewhere: FLA's
use_qk_l2norm_in_kernel supports backward only because it launches the standalone l2norm kernels
inside the op rather than fusing them, while every true fusion (Helion varlen, FlashInfer decode)
is forward-only. k keeps its explicit l2norm; this complements the decode-side preprocessing fusion
in the recurrent path (#317, #334) on the chunked prefill side. Raw-q grams run about sqrt(K) hotter,
so the flag assumes the bounded-gate contract; the eps floor matches l2norm's default and the tests
cover zero rows and 1e-3..1e3 row scales against the explicit path.

GB300, six frozen contract shapes, output-only forward, warm CUDA events: removes the 38-60us
standalone q pass per call (l2norm kernel eliminated; +5us sumsq cost inside the o kernel). This
flipped the h96/h64 mixed-shape comparisons against Helion's pretuned linear-attention kernels;
final six-shape geomean time ratio ours/helion = 0.952.

```bash
pytest test/test_kda_fuse_q_l2norm.py test/test_kda*.py
python agent_space/bench_three_way.py
```

stack-info: PR: #345, branch: drisspg/stack/84

Inference callers currently normalize q with a standalone l2norm pass before chunk_kda, which reads
and writes the full [T, H, 128] tensor just to apply a per-row scalar. Every q-dependent term in the
forward is linear in q's row and lands in the same output row, so the norm can be deferred: the
grams stay in the raw-q basis (intra and K3b unchanged), the output kernel accumulates each row's
sum of squares inside its existing k-tile loop, and one 1/||q|| row scale on the accumulated output
completes the normalization. An earlier variant that emitted an rstd tensor from the intra kernel
cost +154us from register pressure in that 1-warp kernel and was rejected. The fused ragged output
launch needs a slightly larger register cap (152 vs 136) for the extra accumulator.

The flag is opt-in and forward-only: it raises when any input requires grad (the saved Aqk stays in
the raw-q basis, so the existing backward would be wrong) and when compiling (the fused route
bypasses the compiler-opaque autograd op). This mirrors the precedent elsewhere: FLA's
use_qk_l2norm_in_kernel supports backward only because it launches the standalone l2norm kernels
inside the op rather than fusing them, while every true fusion (Helion varlen, FlashInfer decode)
is forward-only. k keeps its explicit l2norm; this complements the decode-side preprocessing fusion
in the recurrent path (#317, #334) on the chunked prefill side. Raw-q grams run about sqrt(K) hotter,
so the flag assumes the bounded-gate contract; the eps floor matches l2norm's default and the tests
cover zero rows and 1e-3..1e3 row scales against the explicit path.

GB300, six frozen contract shapes, output-only forward, warm CUDA events: removes the 38-60us
standalone q pass per call (l2norm kernel eliminated; +5us sumsq cost inside the o kernel). This
flipped the h96/h64 mixed-shape comparisons against Helion's pretuned linear-attention kernels;
final six-shape geomean time ratio ours/helion = 0.952.

```bash
pytest test/test_kda_fuse_q_l2norm.py test/test_kda*.py
python agent_space/bench_three_way.py
```

stack-info: PR: #345, branch: drisspg/stack/84

Inference callers currently normalize q with a standalone l2norm pass before chunk_kda, which reads
and writes the full [T, H, 128] tensor just to apply a per-row scalar. Every q-dependent term in the
forward is linear in q's row and lands in the same output row, so the norm can be deferred: the
grams stay in the raw-q basis (intra and K3b unchanged), the output kernel accumulates each row's
sum of squares inside its existing k-tile loop, and one 1/||q|| row scale on the accumulated output
completes the normalization. An earlier variant that emitted an rstd tensor from the intra kernel
cost +154us from register pressure in that 1-warp kernel and was rejected. The fused ragged output
launch needs a slightly larger register cap (152 vs 136) for the extra accumulator.

The flag is opt-in and forward-only: it raises when any input requires grad (the saved Aqk stays in
the raw-q basis, so the existing backward would be wrong) and when compiling (the fused route
bypasses the compiler-opaque autograd op). This mirrors the precedent elsewhere: FLA's
use_qk_l2norm_in_kernel supports backward only because it launches the standalone l2norm kernels
inside the op rather than fusing them, while every true fusion (Helion varlen, FlashInfer decode)
is forward-only. k keeps its explicit l2norm; this complements the decode-side preprocessing fusion
in the recurrent path (#317, #334) on the chunked prefill side. Raw-q grams run about sqrt(K) hotter,
so the flag assumes the bounded-gate contract; the eps floor matches l2norm's default and the tests
cover zero rows and 1e-3..1e3 row scales against the explicit path.

GB300, six frozen contract shapes, output-only forward, warm CUDA events: removes the 38-60us
standalone q pass per call (l2norm kernel eliminated; +5us sumsq cost inside the o kernel). This
flipped the h96/h64 mixed-shape comparisons against Helion's pretuned linear-attention kernels;
final six-shape geomean time ratio ours/helion = 0.952.

```bash
pytest test/test_kda_fuse_q_l2norm.py test/test_kda*.py
python agent_space/bench_three_way.py
```

stack-info: PR: #345, branch: drisspg/stack/84

Inference callers currently normalize q with a standalone l2norm pass before chunk_kda, which reads
and writes the full [T, H, 128] tensor just to apply a per-row scalar. Every q-dependent term in the
forward is linear in q's row and lands in the same output row, so the norm can be deferred: the
grams stay in the raw-q basis (intra and K3b unchanged), the output kernel accumulates each row's
sum of squares inside its existing k-tile loop, and one 1/||q|| row scale on the accumulated output
completes the normalization. An earlier variant that emitted an rstd tensor from the intra kernel
cost +154us from register pressure in that 1-warp kernel and was rejected. The fused ragged output
launch needs a slightly larger register cap (152 vs 136) for the extra accumulator.

The flag is opt-in and forward-only: it raises when any input requires grad (the saved Aqk stays in
the raw-q basis, so the existing backward would be wrong) and when compiling (the fused route
bypasses the compiler-opaque autograd op). This mirrors the precedent elsewhere: FLA's
use_qk_l2norm_in_kernel supports backward only because it launches the standalone l2norm kernels
inside the op rather than fusing them, while every true fusion (Helion varlen, FlashInfer decode)
is forward-only. k keeps its explicit l2norm; this complements the decode-side preprocessing fusion
in the recurrent path (#317, #334) on the chunked prefill side. Raw-q grams run about sqrt(K) hotter,
so the flag assumes the bounded-gate contract; the eps floor matches l2norm's default and the tests
cover zero rows and 1e-3..1e3 row scales against the explicit path.

GB300, six frozen contract shapes, output-only forward, warm CUDA events: removes the 38-60us
standalone q pass per call (l2norm kernel eliminated; +5us sumsq cost inside the o kernel). This
flipped the h96/h64 mixed-shape comparisons against Helion's pretuned linear-attention kernels;
final six-shape geomean time ratio ours/helion = 0.952.

```bash
pytest test/test_kda_fuse_q_l2norm.py test/test_kda*.py
python agent_space/bench_three_way.py
```

stack-info: PR: #345, branch: drisspg/stack/84